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SECTION  I 

INTRODUCTION 


The  Air  Force  Armament  Laboratory  (AFATL)  is  developing  autonomous  seekers  to 
attack  both  mobile  land  targets  and  fixed,  high-value  land  targets.  The  purpose  of  the 
mathematical  structure  described  in  this  report  is  to  aid  the  efficient  development  of  such 
seekers.  This  mathematical  structure,  known  as  the  AFATL  Image  Algebra,  has  been 
specifically  designed  for  the  concise  expression  and  clear  representation  of  image  processing 
and  pattern  recognition  techniques.  This  structure  provides  a  common  mathematical 
environment  for  target  detection,  algorithm  development,  optimization,  comparison,  coding, 
and  performance  evaluation.  In  addition,  the  Image  Algebra  provides  a  mathematical  basis 
for  a  universal  image  processing  language  which,  when  properly  implemented,  will  greatly 
increase  a  researcher’s  productivity  as  programming  tasks  required  to  compute  image 
transformations  are  greatly  simplified  due  to  the  replacement  of  large  blocks  of  code  by  con¬ 
cise  algebraic  expressions. 

Several  previous  attempts  to  develop  a  unified  algebraic  approach  to  image  processing 
have  met  only  partial  success  in  expressing  all  transformations  of  gray  value  images,  Refer¬ 
ence  1.  In  contrast,  in  addition  to  meeting  the  design  specifications  mentioned  in  the  previ¬ 
ous  paragraph,  the  AFATL  Image  Algebra  provides  a  complete  unified  algebraic  structure 
capable  of  expressing  all  image-to-image  transformations.  The  Image  Algebra’s  foundation 
evolved  from  a  33-month  Air  Force/Defense  Advanced  Research  Project  Agency  (DARPA) 
sponsored  research  effort  known  as  the  Image  Algebra  Project.  Two  contracts  to  develop  the 
Image  Algebra  were  awarded.  One  award  went  to  the  University  of  Florida,  with  Dr. 
Gerhard  Ritter  as  principal  investigator,  and  the  other  to  the  Singer-Kearfott  Corporation 
with  Dr.  Charles  Giardina  as  principal  investigator. 

The  University  of  Florida’s  Image  Algebra  development  proved  highly  successful,  capa¬ 
ble  of  fulfilling  the  tasks  set  forth  by  the  Air  Force  Armament  Division.  In  order  to  further 
enhance  t>e  value  of  the  algebra,  the  Air  Force  Armament  Laboratory  awarded  a  contract 
extension  to  the  University  of  Florida  with  the  goal  of  providing  an  Image  Algebra  Fortran 
implementation  and  to  investigate  a  possible  synthesis  of  the  structures  developed  by  the 
Singer-Kearfott  Corporation  and  University  of  Florida. 


This  report  summarizes  the  University  of  Florida’s  analysis  of  the  two  algebraic  struc¬ 
tures  and  provides  a  tutorial  overview  of  the  AF ATL  Image  Algebra. 


2 


SECTION  n 

ANALYSIS  OF  SINGER-KEARFOTT’S  IMAGE  ALGEBRA 


It  became  clear  at  the  December  1985  DARPA/Center  for  Night  Vision  and  Electro- 
Optics  (CNVEO)  Image  Algebra  briefing  by  the  University  of  Florida  and  Singer- Kearfott 
that  two  vastly  divergent  algebras  were  being  formulated.  The  University  of  Florida’s  alge¬ 
bra  was  viewed  as  the  more  mature  and  superior  of  the  two,  and  the  suggestion  was  made  to 
continue  the  effort  with  the  University  of  Florida  as  sole  contractor.  However,  AFATL  pro¬ 
gram  management  decided  to  retain  both  contractors. 

One  of  the  major  differences  of  the  two  approaches  was  that  Singer-Kearfott’s  algebra 

had  an  infinite  number  of  operators  while  the  University  of  Florida’s  had  10.  After  this 

meeting,  Singer-Kearfott’s  operands  and  operators  started,  to  some  extent,  mimicking  the 

operands  and  operators  established  at  the  University  of  Florida.  However,  as  the  University 

of  Florida’s  operands  and  operators  became  more  refined,  many  of  Singer-Kearfott’s 

operands  and  operators  retained  much  of  the  appearance,  characteristics,  and  properties  of 

several  of  the  University  of  Florida’s  earlier  defined  operands  and  operators.  In  particular, 

Singer-Kearfott’s  images  are  elements  of  X  **  U  RA,  where  Z2  denotes  the  infinite  planar 

A€Z* 

array  with  integral  coordinates,  R  the  set  of  real  numbers,  and  RA  denotes  the  set  of  all 
functions  from  A  to  R.  As  we  will  show  later  on,  this  was  the  University  of  Florida’s  early 
definition  of  real  valued  images.  On  the  other  hand,  images  in  the  University  of  Florida’s 
algebra  are  elements  of  Fx,  where  X  CRk,  Rk  denotes  euclidean  k-dimensional  space,  and  F 
a  set  of  values  such  as  the  set  of  real  numbers,  complex  numbers,  binary  numbers  of  fixed 
length  k,  etc.  This  allows  for  complex  images  as  well  as  discretized  computer  images.  As 
Singer-Kearfott’s  images  do  not  include  complex  imagery,  we  do  not  see  how  any  Fourier-like 
image  transform  can  be  accomplished  within  their  setting. 

As  two  Singer-Kearfott  images  need  not  have  the  same  underlying  array  A,  a  simple 
operation  such  as  the  addition  of  two  images  becomes  a  complicated  issue,  both  in  terms  of 
definition  as  well  as  possible  software  implementation.  Specifically,  the  fundamental  opera¬ 
tors  in  the  Singer-Kearfott  algebra  are: 
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I 

Addition 

© 

n 

Multiplication 

© 

m 

Maximum 

© 

IV 

Division 

© 

V 

Translation 

T 

VI 

Rotation 

N 

vu 

Reflection 

D 

vm 

Domain  Extractor 

K 

DC 

Parameter  Extractor 

G 

X 

Existential  Operator 

E 

Singer-Kearfott’s  image  addition  ©  is  defined  as: 


(f©g)  = 


f(x,y)  (x,y)  €  A  -  B 

g(x,y)  (x,y)  €  B  -  A 

f{x,y)  +  g(x,y)  (x,y)  6AHB 

undefined  (x,y)  ^ AUB 


Image  multiplication  O,  maximum  ©,  and  division  ©  are  defined  in  an  analogous  fashion 
with  4-  being  replaced  by  •,  max,  and  + ,  respectively,  in  the  above  definition. 

1.  REMARKS  AND  OBSERVATIONS 

We  now  consider  each  of  the  basic  operations  of  the  Singer-Kearfott  algebra  and  discuss 
them  in  the  context  of  image  processing  applications  and  the  corresponding  operations  of  the 
University  of  Florida  algebra. 

a.  Image  Addition,  Maximum,  and  Division 

Image  addition,  maximum,  and  division  constitute  the  basic  binary  image  opera¬ 
tions  in  the  Singer-Kearfott  algebra.  The  University  of  Florida  had  defined  this  type  of 
image  addition,  multiplication,  maximum,  and  division  at  the  beginning  of  Phase  I  of  the  pro¬ 
ject.  The  next  page  shows  a  viewgraph  dating  from  the  2nd  project  review  of  Phase  I 
(March  27,  1985). 
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Viewgraph  of  the  2nd  project  review  of  Phase  I  (March  27,  1985): 


I MAGE  ALGEBRA 


THE  OPERATIONS  +.  *.  v.  A  ON  S. 


LET  A  BE  AN  IMAGE  QN  AND  B  BE  AN  IMAGE  ON  N^,  WHERE  N^,N£e  P(X) 


1.  A  +  B  *  C  <x ,  c (x )  )  :  c<x)  “  a(x>  +  b(x)  f  x€  A^IJB^ 


2  2 <2  2 


2  2 
2  2 


2  2 
2  2 


2  2  2  2 
2  2  2  2 
A 


3  4  5  6 

«•  -  *  -i 

2  2 ' 5  6 ' 5  6 


2  2,5  6,4  5 
2  2,5  5 !  4  5 

•  •  •  •  •  J 
2  2  2  2 

2  2  2  2 

A  +  B 


SIMILARLY 

2.  A  *8*  C  <x,c(x))  :  c  <x)  =  a  <x>  *  b  (x)  ,  x  e  Y  PiY  >  l,J  A  KJ  BA 

X  2  B  A 


3.  A  v  B  *  <  (x,  c  (x)  )  :  c  <x )  =  a  <x )  v  b  <x  >  f  x  e  Y  P\Y  >  U  AQkJ  B. 

1  ^  BA 


4.  A  a  B  **  <  (x,c(x)>  :  c  <x )  ■  a  <x)  a  b  (x  >  ,  x  e  Y^fVY^  *  ^  ®a 
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b.  Critique  of  Basic  Binary  Operands 

The  University  of  Florida  abandoned  these  definitions  fairly  early  as  they  are  too 
clumsy  from  both  the  application  and  theoretical  point  of  view  and  the  image  processing 
community  simply  does  not  add  images  this  way.  In  addition,  there  exist  no  natural  additive 
and  multiplicative  identities  for  these  operations  (i.e.  a  0  and  a  1  image).  The  only  identity 
that  works  for  both  is  the  empty  set.  This  does  not  provide  for  a  very  useful  mathematical 
structure. 

c.  Corresponding  University  of  Florida  Operations 

The  Singer-Kearfott  addition  (  multiplication,  maximum,  division  ),  if  ever 
needed,  cam  be  easily  expressed  in  terms  of  the  University  of  Florida’s  current  addition  (mul¬ 
tiplication,  maximum,  division)  followed  by  the  extension  operator. 

d.  Image  Rotation 

Singer-Kearfott’s  rotation  is  defined  only  for  multiples  of  90  degrees  about  the 
origin.  In  the  University  of  Florida’s  algebra  this  can  be  accomplished  through  image 
definition  (  i.e.  the  assignment  statement  a(j,-i)  =  a(i,j))  or  with  a  template  operation. 
Furthermore,  rotation  using  a  template  is  a  one-step  operation  and  images  can  be  rotated 
through  any  angle  and  about  any  point. 

e.  Image  Translation  and  Reflection 

The  operations  erf1  translation  and  reflection  as  defined  in  the  Singer-Kearfott 
algebra  can  again  be  accomplished  by  a  simple  assignment  statement  or  a  one  step  template 
operation.  Furthermore,  Singer-Kearfott’s  translation  is  a  simple  linear  coordinate  shift 
while  translation  by  templates  encompasses  both  linear  and  non-linear  translations. 

f.  Domain  Extractor  and  Parameter  Extractor 

The  domain  extractor  and  parameter  extractor  correspond  to  the  University  of 
Florida’s  operation  of  domain  and  range,  respectively. 

g.  Existential  Operator 

In  regard  to  the  existential  operator,  we  prefer  to  adhere  to  standard  textbook 
mathematics  and  leave  it  as  a  function  definition  statement  (e.g.  "Let  a  be  the  function 
defined  by...")  instead  of  viewing  it  as  an  algebraic  operation.  Our  view  is  identical  with 
computer  language  implementations  in  general.  Functions  that  do  not  exist  in  a  particular 
language  are  defined  by  assignment  statements  (e.g.  a(i,j)  =  i+j,  etc.). 

h.  Remaining  Operators 

The  remaining  operators  consists  of  an  infinite  number  of  macro  operators 
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(subroutines)  which  are  defined  in  terms  of  the  10  elementary  operations.  This  approach 
makes  even  the  simplest  algorithm  appear  extremely  unreadable  and  cumbersome  to  express. 
Two  examples  should  suffice. 

(1)  Morphological  Operations 

The  simple  operations  of  dilation,  erosion,  opening  and  closing  have  the  fol¬ 
lowing  ungainly  appearance: 

[6i(f,8)KU)“E(v0{X(f,Sj)},(ij)] 

| tft (f,  g) ](ij)  -  E| V„ { A(I, e&j) ),  (i,j)] 

6P(f,g)=eft[Di{f,N2(g)},N!(g)] 

et(f,g)-Di[eft(r,s),S| 

where 

V.W-G[  V A5[T_i>_j(f), {(0,0)}]] 

(u)£A 

^(f,g)=©[®([feg|©|iArB©oAUS])|©oArB 

Sff.B)  =*M(f,lB),  and  M(f,g)  =  <(rog,oArB) 

In  addition,  g  is  assumed  to  be  symmetric. 

(2)  The  Gradient  Operator 

The  explanation  of  the  common  gradient  operator  is  as  follows: 

EJfMN.t)  =  Tt[N00  (*(f,M) ,  7(f,N) }  ] 

EMN,t)  =  Tt(N1{/(f,M),^f,N)}] 

EWt)  =  Tt[N2{^f,M),/(f,N)}]  | 

In  each  of  the  above,  M  and  N  are  images  which,  in  practice  represent  directional  masks. 
Here 

fUf„r* . y  -  IfJ 
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N,(f„fs, . U-SIfJ 


k-1 


N#„f2 . fJ-(S[r,|2)2 


Wg)  =  E  ( E  |  P„  [  Z ((,Tu(g) ), T,j(g)  | ,  «i.j) }  | 

(U)€A 


where  S  denotes  ©  addition, 

(iJ)6A 

W,g)-S0|fOg], 

r0(o»[(r@oA)@a(r®oA)r 


T,-r0(fetAU), 


S^f)=o[  S  siW0,{(o,o)}ll 

(ij)6A 

®of  =  (@f)©0A 
where  0A=E[0,K(f)] 


and 


aAf  =  E(a,K(f))Of. 


2.  SUMMARY 

Overall,  we  found  the  Singer-Kearfott  algebra  extremely  cumbersome  and  ungainly,  not 
in  step  with  much  of  current  image  processing  practice.  The  algebra  is  extremely  difficult  to 
use  in  expressing  even  the  simplest  algorithm,  let  alone  an  algorithm  of  any  major  complex¬ 
ity.  This  defeats  at  least  one  major  goad  of  AFATL’s  image  processing  language  develop¬ 
ment  project,  namely  simplicity  and  translucency  of  algorithm  representation. 

In  all  fairness  to  the  Singer-Kearfott  effort  we  must  add  that  our  conclusions  are  based 
on  very  scant  reference  material.  Our  reference  material  consisted  of  Singer’s  Phase  I  final 
report  and  the  Phase  II  final  review  viewgraphs.  Singer-Kearfott’s  final  report,  which  was 
requested  by  us  and  the  intended  basis  of  this  analysis  and  synthesis,  was  never  forwarded  by 
AFATL.  Both,  Singer’s  Phase  I  final  report  and  the  Phase  II  final  review  viewgraphs  are 
sorely  lacking  algorithmic  image  algebra  expressions  ,  i.e.  image  algebra  pseudo  code  of  algo¬ 
rithms  that  represent  more  than  just  a  simple  transform  or  maximum  operation. 

In  summary,  the  University  of  Florida’s  image  algebra  is  capable  of  expressing  Singer- 
Kearfott’s  elemental  operators  in  a  straight  forward  fashion  and  thus  encompasses  all  capa¬ 
bilities  of  the  Singer-Kearfott  algebra.  In  addition,  considering  the  shortcommings  of  the 
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Singer-Kearfott  algebra,  we  feel  that  any  synthesis  of  the  two  algebraic  structures  would  be 
counter-productive  to  the  goals  of  the  AF ATL  image  processing  language  development  pro¬ 
ject.  We  therefore  recommend  the  algebraic  structure  developed  by  the  University  of 
Florida  to  represent  the  AFATL  standard  image  algebra. 
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SECTION  ffl 
IMAGE  ALGEBRA 


In  the  remaining  part  of  this  document  we  provide  an  overview  of  the  AFATL  Standard 
Image  Algebra.  We  begin  by  introducing  the  basic  operands  and  operators  of  the  algebra. 

1.  VALUE  SETS  AND  COORDINATE  SETS 

The  image  algebra  defined  in  this  paper  is  capable  of  multivalue  image  manipulation  and 
constitutes  a  heterogeneous  algebra  in  the  sense  of  Birkhoff,  Reference  2.  This  general  form 
of  the  image  algebra  is  discussed  in  the  last  sections  of  this  document.  In  this  section  we  res¬ 
trict  ourselves  to  single-valued  image  manipulation. 

The  image  algebra  deals  with  six  basic  types  of  operands,  namely  value  sets,  coordinate 
sets,  the  elements  of  each  of  these  sets,  images,  and  templates.  The  only  value  sets  con¬ 
sidered  in  this  section  are  the  set  of  integers,  real  numbers,  extended  real  numbers  (which 
include  the  symbols  +00  and  —00),  complex  numbers,  and  binary  numbers  of  fixed  length  k, 
which  will  be  denoted  by  Z,  R,  RMI  C,  and  Z^,  respectively.  These  correspond  to  the 
values  commonly  encountered  in  most  image  processing  routines  and  in  the  modeling  of  such 
routines. 

An  unspecified  value  set  will  henceforth  be  denoted  by  F.  The  operations  on  and 
between  elements  of  a  given  value  set  F€{Z, R.R,*,,  C,  Z^}  are  the  usual  elementary 
operations  associated  with  F.  Thus,  if  F  =  R,  then  the  operations  are  the  usual  arithmetic 
and  logic  operations  of  addition,  multiplication  and  maximum,  and  the  complementary 
operations  of  subtraction,  division,  and  minimum.  In  addition  to  these  elementary  operations 
on  elements  of  value  sets,  the  image  algebra  also  includes  the  operations  of  union,  intersec¬ 
tion,  set  subtraction,  choice  function,  and  cardinality  function  on  subsets  of  F.  The  choice 
function  applied  to  a  set  returns  (chooses)  an  arbitrary  element  of  the  set,  while  the  cardinal¬ 
ity  function  yields  the  number  of  elements  in  the  set.  Union,  intersection,  set  subtraction, 
choice  function,  and  cardinality  function  will  be  denoted  by  U,  D,  \  ,  choice,  and  card, 
respectively. 

Coordinate  sets  are  subsets  of  n-dimensional  Euclidean  space  Rn.  We  reserve  the  letters 
X,  Y,  and  W  to  denote  coordinate  sets.  Elements  of  coordinate  sets  will  be  denoted  by  bold 
lower  case  letters.  In  particular,  if  x€X  and  XCR",  then  x  is  of  form 
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x=(x1>x2,  •  •  •  ,xn),  where  each  coordinate  Xj  (i  =  1,2,  ...  ,n)  is  a  real  number. 

It  follows  from  the  definition  that  coordinate  sets  can  be  rectangular,  hexagonal,  toroidal 
discrete  arrays  as  well  as  infinite  subsets  of  R“.  Providing  coordinate  sets  with  such  wide 
varieties  of  shapes,  sizes  and  dimensions  allows  for  a  coherent  mathematical  approach  to  the 
modeling  and  manipulation  of  continuous  as  well  as  discrete  images  on  any  desired  type  of 
coordinate  set. 

Image  algebra  operations  acting  on  coordinate  sets  are  operations  on  subsets  of  coordi¬ 
nate  sets  as  well  as  operations  between  coordinate  points.  In  particular,  operations  on  sub¬ 
sets  of  coordinate  sets  are  U,  D,  \  ,  choice  function,  and  cardinality  function.  Image  algebra 
operations  on  or  between  elements  of  coordinate  sets  are  the  usual  operations  between  coor¬ 
dinate  points,  i.e.,  vector  addition,  scalar  and  vector  multiplication,  dot  product,  etc. 

2.  IMAGES 

Thus  far  we  have  defined  two  types  of  objects,  value  sets  and  coordinate  sets.  These 
sets  and  their  elements  constitute  some  important  operands  of  the  image  algebra.  However, 
the  most  fundamental  of  the  algebra’s  operands  are  images.  The  most  general,  yet  useful, 
mathematical  definition  of  an  image  involves  the  previously  defined  concepts  of  value  sets 
and  coordinate  sets. 

Given  a  coordinate  and  value  sets  X  and  F,  respectively,  then  an  F  Ivalued  image  a  on 
X  is  the  graph  of  a  function  a:X  — ♦  F.  Thus,  an  F  valued  image  a  on  X  is  of  the  form: 

a  =  {(x,a(x)):x€X}, 

where  a(x)  €  F. 

The  set  X  is  called  the  set  of  image  coordinates  of  a,  and  the  range  of  the  function  a 
(which  is  a  subset  of  F)  is  the  set  of  image  values  of  a.  An  element  (x,a(x))  of  the  image  a  is 
called  a  picture  element  or  pixel,  where  x  is  called  the  pixel  location,  and  a(x)  the  pixel  value 
at  location  x.  The  set  of  all  F  valued  images  on  X  is  denoted  by  Fx.  Here  we  follow  the 
usual  mathematical  convention  of  denoting  the  set  of  all  functions  from  a  set  A  to  a  set  B  by 
Ba. 

If  the  value  set  F  —  Z  or  F  =  R,  then  we  are  dealing  with  integer  or  real  valued 
images,  respectively.  Similarly,  replacing  F  by  C,  or  Z^,  provides  for  complex  or  finite  digi¬ 
tal  images,  respectively. 
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In  the  next  chapter  we  extend  the  notion  of  value  sets  in  order  to  include  such  objects  as 
vector  valued  images.  This  fact  and  the  above  examples  should  make  it  clear  that  the  vari¬ 
ous  choices  for  F  and  X  allow  for  a  far  greater  variety  of  image  operands  than  are  currently 
used  by  the  image  processing  community. 


3.  BINARY  AND  UNARY  OPERATIONS  ON  IMAGES 

Operations  on  and  between  F  valued  images  are  the  natural  induced  operations  of  the 
algebraic  system  F.  For  example,  for  real  valued  images  (i.e.  elements  of  Rx),  the  operar 
tions  are  the  elementary  operations  induced  by  the  vector  lattice  (a  vector  space  which  is 
also  a  lattice)  R.  Thus,  the  basic  real  valued  image  operations  reflect  the  arithmetic  and 
logic  operations  on  R.  In  particular,  the  binary  operations  of  addition,  multiplication,  and 
maximum  on  Rx  are  defined  as  follows: 

Let  a,  b  €  Rx.  Then 

a  +  b  =  {  (x,e(x))  :  c(x)  =  a(x)  +  b(x),  x  €  X}  (1) 

a  *  b  =  {  (x,c(x))  :  c(x)  =  a(x)  •  b(x),  x  6  X)  (2) 

% 

*  V  b  s  { (x,e(x)) :  c(x)  -  a(x)  V  b(x),  x  €  X}  (3) 

These  are  the  basic  binary  operations  for  real  valued  images.  As  complex  numbers  are 
not  endowed  with  a  "natural”  lattice  structure,  only  operations  I  and  2  are  basic  operations 
between  complex  valued  images. 

Analogous  to  the  development  of  the  algebra  of  real  numbers,  other  binary  and  unary 
operations  on  real  valued  images  can  now  be  derived  from  the  basic  operations  either  directly 
or  in  terms  of  series  expansion.  However,  instead  of  reinventing  the  wheel,  we  assume  fami¬ 
liarity  with  the  algebra  of  real  numbers  and  let  the  remaining  operations  on  Rx  again  be 
induced  by  the  corresponding  operations  on  R.  Two  of  these  operations  -  commonly  used  in 
image  processing  -  are  exponentiation  and  the  computation  of  logarithms.  In  particular,  if  a 
and  b  are  real  valued  images  on  X  then 

ab  =  {(x,c(x))  :  c(x)  =  a(x)bM  if  a(x)  /  0,  otherwise  c(x)  =  0,  x  €  X  }.  (4) 

As  we  are  dealing  with  real  valued  images  we  follow  the  rules  of  real  arithmetic  and  res¬ 
trict  this  binary  operation  to  those  pairs  of  images  a,b  for  which  a(x)b^x'€R  whenever 
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a(x)#0.  This  prevents  the  creation  of  complex  pixel  values  such  as  (— 1)*.  The  inverse  of 
exponentiation  is  defined  in  the  usual  way  by  taking  the  logarithm,  namely 

logab  &  {(x,c(x))  :  c(x)  =  l°ga(x)b(x),  x  e  X  } .  (5) 

As  for  real  numbers,  logab  is  defined  only  for  those  images  a  and  b  for  which  a(x)  >  0 
and  b(x)  >  0  for  all  x  €  X.  The  next  basic  binary  operation,  called  the  dot  product,  distin¬ 
guishes  itself  from  the  above  five  in  that  its  output  is  not  an  image  but  a  real  number.  Let 
X  be  finite,  then  the  dot  product  is  defined  as: 

A#b  S  (6) 

An  image  a  is  called  a  constant  image  if  all  its  gray  values  are  the  same;  i.e.  if  a(x)  =  k 
for  some  real  number  k  and  for  all  x  €  X 

Two  constant  images  are  of  utmost  importance  in  the  image  algebra;  these  are  the  zero 
image,  defined  by  Os{(x,0):x€X},  and  the  unit  image,  defined  by  l={(x,l):x€X}. 

Suppose  k  €  R  and  a  is  a  constant  image  with  a(x)  =  k.  Then  we  define: 
bk  s  b*  and  kb  =  ab 
kb  S  a*b  and  k  +b  =  a  +  b 
logkb  3  logab. 

In  the  definition  of  log  we  assume,  of  course,  that  k>0  and  b(x)>0  for  all  x.  We  also  note 
that  exponentiation  is  defined  even  when  a(x)  =0. 

Subtraction,  division  and  minimum  are  defined  in  terms  of  the  basic  operations  and 
inverses.  Specifically: 

a  —  b  =  a  +  (— b)  and  a/b  =  a*b-1,  where  — b  =  {(x,— b(x)):  (x,b(x))€b  } 
a  A  b  s  —(—a  V  — b) 

The  images  0  and  1  have  the  obvious  property  a+0  =  a  and  a*l  =  a.  On  the  other 
hand,  b*b-1  does  not  necessarily  equal  1.  However,  b*b-1*b  =  b.  For  this  reason  b"1  is 
called  the  pseudo  inverse  of  b.  Inequalities  between  images  are  defined  in  terms  of  maximum 
and  minimum.  Thus,  for  example,  a<b  if  and  only  if  aVb=b.  These  observations  show 
that  the  ring  (Rx,+,*)  and  the  lattice  (RX,V,A)  behave  very  much  like  the  ring  and  lat¬ 
tice  of  real  numbers. 
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There  are  various  useful  unary  operations  definable  in  terms  of  the  basic  binary  operar 
tions.  For  example,  we  have  already  provided  the  definitions  of  kb  and  logkb,  the  exponen¬ 
tial  of  an  image  and  the  logarithm  of  an  image  b  to  the  base  k,  respectively.  In  particular, 
the  exponential  of  an  image  b,  exp(b)  =  eb  =  ab,  and  the  natural  logarithm  of  b  is  defined 
as  lnbslog^b,  respectively,  where  a  is  the  constant  image  defined  by  a(x)  =  e  for  all  x6X. 
Similarly,  the  absolute  value  of  an  image  a  can  be  defined  by  I  al  =aV(— a). 

In  view  of  these  examples  it  is  obvious  that  there  are  various  operations  on  and  between 
images  of  different  degrees  of  complexity  that  can  be  derived  from  the  operations  defined 
thus  far.  Phrasing  some  of  these  in  terms  of  the  basic  operations  would  result  in  complicated 
algebraic  expressions  or  infinite  series  representation.  However,  this  would  defeat  the  goal  of 
providing  a  simple  language  for  image  processing  tasks.  Instead,  we  follow  our  initial  philo¬ 
sophy  and  define  the  operations  on  Fx  to  be  the  operations  induced  by  the  (usually  well- 
known)  operations  of  the  algebraic  system  F.  In  particular,  the  common  unary  operations  on 
Rx  are  functions  available  in  most  high  level  programming  languages.  More  generally,  any 
function  /:  R  — ►  R  induces  a  function  Rx  — +•  Rx,  again  denoted  by  /,  and  defined  by 

/(»)  -  {(x,c(x)):c(x)  =  /(a(x))} 

For  example,  sin(&)  <*  {(x,sin(a(x))):  x  €  X  }.  Similarly,  if  denotes  the  characteristic  func¬ 
tion  with  respect  to  some  set  SCR,  then 

Xs(a)  **  {(*.>(isWx))):*€X}  **  {  (x,c(x))  :  c(x)  =  l  if  a(x)€S,  otherwise  c(x)  =  0  }. 

Excepting  the  formalism,  many  of  the  operations  of  the  algebraic  system  Rx  described 
thus  far  are  not  new,  and  are  well-known  to  the  image  processing  community.  What  is  new 
is  the  concept  of  raising  an  image  to  the  power  of  another  image  or  taking  the  logarithm  of 
an  image  to  the  base  of  another  image.  This  generalizes  the  common  pixel  level  operations 
of  raising  an  image  to  a  constant  such  as  taking  the  square  root  of  an  image  at  each  pixel  or 
taking  the  natural  logarithm  of  an  image  at  each  pixel. 

The  idea  of  having  more  versatile  pixel  level  operations  has  led  to  the  following  generali¬ 
zation  of  the  characteristic  function:  Given  a€Fx  and  S€(2F)X,  where  2F  denotes  the 
power  set  of  F  so  that  S(x)CF  for  each  xEX,  then 

>(s(a)  =  {(x,c(x))  :  c(x)  =  l  if  a(x)€S(x),  otherwise  c(x)  =  0  }.  (7) 

Pixel  level  image  comparison  provides  a  simple  application  example  of  the  generalized 
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characteristic  function.  Given  the  image  b€Rx,  we  define  S<bG(2R)x  by 
S<b(x)  =  {  r  €  R :  r  <b(x) }.  The  functions  S<b,  S^,  S^,  and  S>b  are  defined  analogously. 
Thus,  for  example,  S>b(x)={r€R:r  >  b(x)}  and  S_b(x)  =  {r€R:r=b(x)}.  Substituting 
these  set  functions  for  S  in  Equation  (7)  yields: 

Xs-Ja)  *  ( (x-c(x))  :  c(x)  =  1  if  a(x)>Mx)i  else  c(x)=0  }, 

XsJ,*)  =  (  (x>c(x))  :  c(xH  ^  a(x)<b(x),  else  c(x)=0  },  etc. 

In  order  to  reduce  and  simplify  notation,  we  define  x>b  =  Xs>b,  X<b  =  Xs.^  X<b  =  Xs^ , 
X>b  =  Xs^i  Xb  =  Xs^-  As  alluded  to  earlier,  these  generalized  characteristic  functions 
could  have  been  defined  in  terms  of  the  more  elementary  image  operations.  Specifically,  we 
could  have  defined 

X>b(a)  *  [(a—b)  V  0]-1  *  ((a— b)  V  0] 

Obviously,  if  a=b,  then  (a—b)  VO  =  0  and  X>b  (a)  ~  0-1*0  =  0  since  by  definition  of 
exponentiation,  0_1  =  0.  The  function  X<b  Is  defined  in  a  similar  fashion.  The  remaining 
characteristic  functions  that  compare  two  images  can  then  be  defined  in  terms  of  products 
and  complementations.  In  particular,  if  we  define  the  complement  a  of  an  image  a  by 
a  =  1  —  a*a~l,  then  we  obtain 

X<b(a)  «X>b(a).  x>b(a)  *x<b(a)>  and  Xi,(a)  =  X<b(a)  *  X>b(a)  ■ 

Whenever  b  is  the  constant  image  with  gray  values  equal  to  k  it  is  customary  to  replace 
b  by  lc  in  the  above  definitions.  Figure  I  below  provides  an  example  of  the  operation  X>k(a), 
where  k  =  15.  In  a  likewise  fashion  we  can  retain  pixels  whose  values  are  within  an  interval 
[m,n]  by  using 

Xjm,n|(a)  =  X>m(a)  *  X<n(a)  • 
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Figure  1.  Example  of  the  Operation  X>15(a) 

When  considering  an  expression  of  the  form  sin(a),  one  rarely  thinks  of  the  image  a  as  a 
function.  However,  images,  as  defined  in  this  report,  are  functions,  namely  elements  of  Fx, 
and  several  important  mathematical  notions  used  in  image  processing  are  the  restriction, 
extension,  domain,  and  range  of  a  function.  We  express  these  notions  as  basic  operations  of 
the  image  algebra  In  particular,  if  a  is  an  image  on  X,  then  the  Domain  function  of  an 
image  a  is  defined  simply  as 

Domain(a)  =  set  over  which  a  is  defined, 

e.g.,  Domain(a)  *X  The  range  function  of  an  image  lies  on  the  other  side  of  the  spectrum, 
and  it  provides  the  set  of  values  assumed  by  the  image.  Thus,  if  a  E  Fx,  then 

Range(a)  =  set  of  values  determined  by  a, 

e.g.,  Range(a)  C  F  is  the  set  of  all  values  a  assumes  on  X  Therefore,  the  output  of  domain 
or  range  is  not  an  image  array  but  a  set  of  coordinate  points  or  a  set  of  values,  respectively. 

Let  a  €  Fx.  The  restriction  of  a  to  a  subset  Y  of  X  is  denoted  by  a|Y.  Thus,  a|Y  €  FY 

Here  a  user  would  specify  the  coordinate  set  Y  C  X  As  an  example,  he  could  set  Y  = 
{x€X  :  5  <lxl  <20  }.  We  also  allow  the  restriction  of  a  to  a  subset  of  X  specified  by  an 
image-dependent  property  such  as  Y  =  {x€X:a(x)G  S},  where  S  C  F.  This  type  of  res¬ 
triction  is  denoted  by  a  double  vertical  bar  and  provides  a  useful  tool  for  expressing  various 
algorithmic  procedures.  For  example,  if  Y  =  {x€X:a(x)  >  T },  where  T  denotes  a  given 
threshold  value,  then  we  define  ai  by  a»  s  ai,  _  In  this  case,  note  that 

">T  ">T  >TJ 


Domain(a(>r)  is  the  set  of  all  locations  where  a  exceeds  the  threshold  T. 

Let  a  be  an  image  on  X,  b  an  image  on  Y,  and  X  C  Y.  The  extension  of  a  to  b  on  Y 
is  defined  by 


a|(b-Y><x)  = 


ja(x)  ifxGX 
jb(x)ifxGY\X, 


where  Y\X  =  {y  €Y :  y  $:X}.  In  actual  practice,  the  user  will  have  to  specify  the  func¬ 
tion  (image)  b  on  Y. 

4.  GENERALIZED  TEMPLATES 

In  terms  of  image  processing  applications,  templates  and  template  operations  are  the 
most  powerful  tool  of  the  image  algebra.  The  image  algebra  definition  of  a  template  unifies 
and  generalizes  the  usual  concepts  of  templates,  masks,  windows  and  neighborhood  functions 
into  one  general  mathematical  entity.  In  addition,  templates  as  defined  in  this  report  gen¬ 
eralize  the  notion  of  "structuring  elements"  as  used  in  mathematical  morphology,  Refer¬ 
ence  3. 

Let  X  and  Y  be  coordinate  sets  and  F  a  value  set.  A  generalized  F  valued  template  t 
from  Y  to  X  is  a  function  t:Y— *  Fx.  Thus,  for  each  y  G  Y,  t(y)  G  Fx,  or,  equivalently, 
t(y)  is  an  F  valued  image  on  X.  The  set  Y  is  called  the  target  array,  and  the  set  X  is  called 
the  source  array  of  the  template  t.  For  notational  convenience  we  define  ty  =  t(y).  Thus, 
ty  =  {(x,ty(x)):x€X}.  The  point  y  is  called  the  target  point  of  the  template  t,  and  the 
values  ty(x)  are  called  the  weights  of  the  template  t. 

The  set  of  all  F  valued  templates  from  Y  to  X  will,  henceforth,  be  denoted  by  (FX)Y. 
If  t  is  a  real  valued  template  from  Y  to  X  then  we  define  the  support  of  ty  to  be  S(ty)  =  {  x 
G  X  :  fcy(x)  0  }.  If  t  is  an  extended  real  valued  template  then  we  also  define  S^ty)  =  {  x 
G  X  :  ty(x)  ^  d*x>  }.  The  sets  $(ty)  and  S^ty)  are  also  referred  to  as  the  configuration  of  t 
at  target  pixel  y  or  simply  as  the  source  configuration.  Figure  2  illustrates  these  concepts. 
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Target  Array  Y  Source  Array  X 


Figure  2.  Pictorial  Example  of  a  Template  from  Y  to  X 

If  t  6  (RXp  or  t  €  (R*)*,  then  t  is  called  translation  invariant  if  and  only  if  for  each 
triple  x,  y,  *  €  X  with  y+*  and  x+i  €  X  we  have  that  ty(x)  =  ty+1(x+*).  A  template 
which  is  not  translation  invariant  is  called  a  translation  variant  or,  simply,  a  variant  tem¬ 
plate.  A  large  class  of  translation  invariant  templates  with  finite  support  have  the  nice  pro¬ 
perty  that  they  can  be  defined  pictorially.  For  example,  let  X  =  Z2,  where 
Z2  =  Z  X  Z  C  R2  is  the  set  of  discrete  lattice  points  with  integral  coordinates.  Let  y  =(x,y) 
be  an  arbitrary  point  of  X  *i  -  (x,y-l),  x2  =  (x+l,y),  and  x3  =  (x+l,y-l).  We  now  define 
a  template  t  €  (R*)*  by  defining  -  for  each  y€X  -  its  weights  as  ty(y)  =  1,  t^Xj)  =  3, 
^y(x2)  =  2,  ty(x3)  =  4,  and  ty(x)  =  0  if  x  is  not  an  element  of  the  set  {y,x1,x2,x3}.  Note  that 
it  follows  from  our  definition  of  t  that  S(ty)  =  {y,Xj,x2,x3}.  Thus  t  has  configuration  and 
weights  as  shown  in  Figure  3: 
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y 


Figure  3.  Pictorial  Example  of  a  Translation  Invariant  Template 
Th«  whftded  sell  in  th«  pictorial  representation  of  t  indicates  the  location  of  the  target  point 

y 

Templates  are  used  to  define  those  image  transformations  which  make  use  of  all  image 
values  within  some  predescribed  configuration  of  the  source  or  input  image.  For  example, 
given  an  image  a  on  X  and  t  a  template  from  Y  to  X,  where  Y  may  be  of  an  entirely 
different  shape,  size  or  dimension  than  X,  then  an  operation  between  a  and  t  will  transform 
a  into  an  image  b  on  Y  where  each  new  pixel  value  b(y)  is  computed  in  terms  of  some  arith¬ 
metic  and/or  logic  combination  of  the  values  a(x)  and  ty(x),  where  x  ranges  over  S(ty)  or 
$»(**)•  Initially  it  may  be  convenient  to  view  templates  as  masks  such  as  the  edge  masks 
used  in  the  Sobel  or  Kirsch  edge  detection  schemes.  It  is  important  to  realize,  however,  that 
the  notion  of  a  template  is  not  the  same  as  that  of  a  mask.  The  magnification  template 
presented  in  this  report  and  Fourier  templates  presented  in  References  4  and  5  are  examples 
of  templates  which  are  not  masks.  In  the  former,  the  target  pixels  are  not  even  members  of 
their  respective  configurations,  while  in  the  latter  the  weights  and  configurations  change  as  a 
function  of  the  position  of  the  target  pixel. 


5.  OPERATIONS  BETWEEN  IMAGES  AND  TEMPLATES 

There  are  three  basic  template  operations  that  awe  used  to  transform  a  real  valued 
image.  They  are  denoted  ©,  EJ ,  and  ©,  and  called  generalized  convolution,  additive 
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maximum,  and  multiplicative  maximum,  respectively.  For  complex  valued  images  only  one 
image- template  operation  is  defined,  namely  ©. 

The  template  operations  compute  a  pixel  value  c(y)  by  performing  the  basic  operation 
of  addition  or  maximum  on  a  weighted  collection  of  pixel  values  a(x),  with  coordinates  x  in 
some  subset  of  X.  Let  XC  R"  be  finite  and  Y  C  Rm.  Suppose  a  G  Rx  and  t  G  ( RX)Y, 
then  we  define 

a®t  =  {(y,e(y)):  c(y)  =  E  a(x)  t  (x),  y  G  Y} 

x€X 

and 

aM  t  =  {(y,c(y)):  c(y)  *  V v»{x) -l-t^x),  y  €  Y}, 


where  V  a(x)  -fty(x)  =  max{  a(x)  -4-ty(x)  :  x  6  X  }. 

Several  important  comments  are  now  in  order.  First  note  that  since  ty(x)  =0  whenever 

x  £  5(ty),  we  have  that  E  a(x)  ty(x)  =  E  a(x)-ty(x).  Here  we  use  the  convention 

x€X  x6  S[ty) 

E^a(x)ty(x)  =  0  whenever  $(ty)  =  0  Thus,  the  new  pixel  value  c(y)  depends  only  on  the 
values  of  a(x)  and  ty(x)  for  x  €  $(ty). 

Similarly,  if  t  G  (RX)Y  and  ty(x)  =  — oo  whenever  x  £  then  V  a(x)  +ty(x)  = 

x€X 

V  a/x)  +t  (x).  Here  we  define  V  a(x)+tv(x)  =  — oo  whenever  SJitv)  =  0.  Hence, 

the  action  for  computing  the  new  pixel  value  c{y)  generally  takes  place  over  the  set  S(ty)  or 
SJty). 

It  may  be  apparent  by  now  that  the  sets  S(ty)  and  SJi ty)  generalize  the  notion  of  a  con¬ 
volution  window  or  mask  and  a  morphological  structuring  element,  respectively.  The  appli¬ 
cation  examples  provided  in  this  paper  should  clarify  the  analogy. 

The  operation  of  multiplicative  maximum  is  analogous  to  that  of  the  additive  maximum. 
In  particular,  for  a  G  Rx  and  t  G  (RX)Y  with  ty(x)=— oo  whenever  x^S^ty),  we  define 

s  {(y,c(y)):  c(y)  =  V  a(x)  t  (x),  y  G  Y} 


where 


V 


a(x)  ty(x)  =  —oo  whenever  Sgj[ty)  = 


0 
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There  is  an  easy  generalization  of  the  above  defined  operations  between  images  and  tem¬ 
plates  that  proves  useful  when  expressing  algorithms  in  image  algebra  code.  Suppose 
XCR",  WCR",  YCRm,  a€Rx,  and  t€(Rw)Y  with  $(ty)  finite  for  each  y  €  Y.  Then  we 
define 

a®t  s  {(y,c(y)) :  e(y)  =  £  a(x)-ty(x),  y  €  Y} , 

x€Xn5(ty) 

where  c(y)  =  0  whenever  X  H  S(ty)  =  0. 

The  operations  Q  and  ©  are  defined  in  a  similar  fashion.  For  example,  for  Q  we 
define  the  new  pixel  value  c(y)  =  V  a(x)  +ty(x) . 

xexnut,) 

The  complementary  operations  of  multiplicative  minimum  and  additive  minimum  are 
defined  in  terms  of  ©  and  S3  as  follows: 

a@t  =  — (a©— t) 
aQt  =  —(—a  E3  — t) 

In  the  above  definitions  we  assume  that  $(ty)  and  S,Jty)  are  finite  for  each  y  EY. 
However,  the  definitions  extend  to  continuous  functions  a(x)  and  ty  on  compact  sets  $(ty) 
and  Sd ty),  with  the  exception  that  in  the  formulation  of  a©t  the  sum  is  replaced  by  an 
integral.  That  is, 

«(y)  =  5£a(x)'Mx)dx 

Thus,  image  algebra  operations  can  be  used  for  expressing  both  continuous  and  discrete 
image  transformations. 

As  a  final  comment  we  note  that  while  a  6  Rx,  a®t,  aQt,  and  a©t  are  all  elements 
of  Ry  (or  Ry).  Thus,  template  operations  may  be  used  for  changing  the  dimensionality  or 
size  and  shape  of  images.  In  particular,  in  addition  to  the  usual  local  or  global  convolutions  - 
as  occur  in  edge  enhancement,  local  smoothing,  morphological  operations  and  Fourier  like 
transformations  -  template  operations  also  provide  a  tool  for  image  rotation,  zooming,  image 
reduction,  masked  extraction,  and  matrix  multiplication. 

At  first  glance,  these  operations  may  appear  somewhat  mysterious.  However,  the  exam¬ 
ples  provided  below  and  those  given  in  References  6,7,8,  and  9  illustrate  the  inherent 
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simplicity  and  power  of  these  operations. 

Example  1.  Local  Averaging.  Let  a  be  an  image  on  a  rectangular  array  X  C  Z2.  Let 
Y  =  Z2  and  t  €  (RY)Y  be  the  3x3  neighborhood  template  defined  as  follows: 


1 

1 

1 

I 

/  s 
1 

v  ✓ 

1 

1 

1 

1 

Then  — a©t  represents  the  image  obtained  from  a  by  local  averaging  since  the  new 


pixel  value  is  given  by  e(y)  =  -jj- 

y 


£  a(x). 

x€S(ty)T)X 


As  an  important  remark,  we  note  that  the  image  a© fc  is  an  image  on  all  of  Z2  with  zero 
values  outside  of  the  array  XC  Z2.  Obviously,  computers  are  not  capable  of  storing  images 
defined  on  infinite  arrays.  Furthermore,  in  practice  one  is  only  interested  in  the  image 

-^-(a©t)  restricted  to  the  array  X,  that  is  “(a®t)lx,  where  lx  denotes  the  restriction  to 
y  y 

X 


This  problem  could  be  solved  as  follows:  Let  s  €  (Rx)x  be  defined  by  sy  =  (ty)lx  for 
each  y  €  X,  where  t  is  the  template  defined  in  Example  1.  Then  -^-a©s  provides  the 

desired  finite  image,  since  (a©t)|x  —  a©s.  Thus,  the  question  arises:  "Why  not  simply 
define  t  as  a  template  from  X  to  X  instead  from  Z2  to  Z2?" 

The  rationale  for  defining  the  template  as  we  did  is  that  this  template  can  be  used  for 
smoothing  any  2-dimensional  image  independent  of  its  array  size  X  The  reason  for  this  is 
that  when  defining  an  image  b  in  a  program  one  is  usually  forced  to  declare  its  dimensions, 
i.e.  the  size  of  its  underlying  array  X.  In  particular,  an  image  algebra  "program”  statement 
of  form  b  =  a©t  means  to  replace  b  pointwise  by  a©t  so  that  the  value  of  b  at  location  y 
is  the  value  of  a©t  at  location  y.  That  is,  the  array  on  the  left  side  of  the  equality  sign 
induces  a  restriction  on  the  right  side  image  array.  In  short,  we  make  the  convention  that 
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the  image  algebra  equation  b  =  (a©t)l  x,  where  X  is  the  domain  of  b,  corresponds  to  the 
image  algebra  program  statement  b  =  a©t.  Thus,  a  programmer  is  not  faced  with  the  task 
of  redefining  t  for  a  different  sized  image,  as  would  be  the  case  if  he  had  defined  t€(Rx)x 
for  a  given  X  In  fact,  this  is  the  way  we  have  embedded  image  algebra  into  image  algebra 
FORTRAN,  Reference  10. 

Of  course,  the  program  statement  b  =  (a©t)  will  produce  a  boundary  effect.  In  par- 

y 

ticular,  if  a  and  b  are  mX  n  images  with  underlying  coordinate  set 
X  =  {(i,j) :  1  <  i<  m,  1  <  j<  n  },  then 

b(l,l)  -  — (a(l,l)  +  a(l,2)  +  3(2,1)  +  a(2,2)) 

which  is  not  the  average  of  four  points.  One  may  either  ignore  this  boundary  effect  or  use 
one  of  several  schemes  to  prevent  it.  For  instance,  one  may  simply  avoid  the  boundary  pix¬ 
els  by  defining  an  array  Y  *■  {  (i,j)  €  X  :  i  £  {l,m},  and  j  £  {l,n}  }  and  the  template 

t  €  (Rx)y  with  ty  having  configuration  and  weights  as  shown  in  Example  1.  Then  -^-(a©t) 

represents  the  desired  (m— 2)X  (n— 2)  output  image.  Letting  m  and  n  be  variables  again 
allows  the  application  of  t  to  any  size  images. 

Example  2.  Sobel  Edge  Detection.  Let  a  be  am  image  on  a  rectangular  array  X  C  Z2,  and 
s,  t  the  templates  shown  below  defined  on  all  of  Z2.  The  image  algebra  expression 

[(a  ©  s)2  +  (a  ©  t)2]1/2 , 

where 
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/  V 
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represents  the  Sobel  edge  enhanced  image.  The  simplicity  and  translucency  of  image  algebra 
expressions  is  apparent.  Local  averaging  and  Sobel  edge  detection  expressed  in  the  image 
algebra  "look  like"  their  corresponding  textbook  formulations,  References  11  and  12.  There 
is  no  lengthy  code  or  obscure  symbology  involved  in  these  expressions. 


23 


Example  3.  Geometric  Edge  Filtering.  This  technique  employs  the  operators  ©  and  ® . 
Let  tlf  tj,  t3,  and  t4  be  the  templates  defined  as  follows: 


rjm 

2 

2 

t2  — 

i 

1 

wK 
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Wa 
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1 

/  \ 

/  \ 

1 

fc4  = 

2 

s.  s 

v  / 

2 

1 

Then  the  image  algebra  expression 

b  =  {(a©tj  —  a®^]2  +  [a©t3  —  a®t4]2}2 

is  an  edge  enhancement  technique  which  enhances  edges  while  smoothing  regions  without 
sharp  edge  contrast.  An  example  is  shown  in  the  figure  below. 
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Figure  4.  (a)  Input  Image  (b)  Edge  Enhanced  Image 

Example  4.  Dilations  and  Erosions.  We  present  this  example  for  readers  familiar  with  the 
basic  notions  of  dilation,  erosion  and  structuring  elements  that  define  all  image  processing 
schemes  based  on  mathematical  morphology.  More  details  of  the  relationship  between  image 
algebra  and  mathematical  morphology  can  be  found  in  Reference  3. 

The  image  algebra  convolution  operators  0  and  0  can  be  used  to  express  the  mor¬ 
phological  operations  of  dilation  and  erosion,  respectively,  for  both  boolean  and  gray  valued 
images.  In  particular,  if  B  denotes  the  structuring  element  used  in  a  dilation  or  an  erosion, 
then  we  define  a  template  t  corresponding  to  B  by  setting  S^ty)  =  By',  where  By'  denotes 
the  reflection  of  By,  and  By  the  translation  of  B  by  the  vector  y.  The  template  weights  ty(x) 
are  defined  to  be  the  values  assigned  to  By'  at  location  x  and  ty(x)  =  —  oo  if  x  £  By.  Then 

I  a  0 1  is  equivalent  to  the  dilation  of  a  by  B,  and 

II  a  0  — t  is  equivalent  to  the  erosion  of  a  by  B. 

Statements  I  and  II  are  true  for  both  boolean  and  gray  level  dilation  and  erosions. 


fu# 


(a)  The  Input  Image  a  (b)  The  Dilated  Image  b=a  0  t  (c)  The  Eroded  Image  b  ES  t 


Figure  5.  Example  of  Dilation  and  Erosion 

Note  that  the  operation  a  0 1  simply  replaces  each  pixel  value  of  a  by  the  maximum  pixel 
value  in  the  configuration  of  t;  i.e.  each  pixel  value  a(x)  is  compared  with  the  values  of  its 
directly  adjacent  horizontal  and  vertical  neighbors  and  the  maximum  value  of  these  five  pos¬ 
sible  pixel  values  replaces  a(x).  Similarly,  for  (aQt)Qt  each  pixel  value  of  a  0 1  is 
replaced  by  the  minimum  pixel  value  in  the  configuration  of  t. 

It  is  important  to  note  that  a  template  may  vary  at  different  locations  in  both  shape  and 
weights.  Thus  the  expression  a  0  t  may  represent  a  far  more  complex  algorithm  than  a 
simple  dilation.  In  short,  mathematical  morphology,  as  used  in  actual  image  processing,  is  a 
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special  substructure  of  the  mathematical  structure  represented  by  the  image  algebra,  Refer¬ 
ence  3. 


Before  discussing  parameterized  templates,  we  introduce  two  special  but  very  important 
image  algebra  operations.  These  are  the  sum  and  the  maximum  of  an  image.  The  sum  of  an 
image  a  on  X  is  defined  as 

Sa  =  Y  a(x) 

x€X 

and  the  maximum  of  a  as 

Va  =  max{a(x):x€X}. 

Of  course,  the  sum  and  maximum  can  be  expressed  in  terms  of  more  elementary  operations. 
Namely, 

Sa  =  a*l  and  Va  s  £(a©t), 

where  t  is  a  template  from  Y  =  {0}  to  X  defined  by  t0(x)  =  1  for  each  x  6  X.  Note  that 

5^a  =  Yj  a(x)  is  a  real  number,  a©t  is  an  image  consisting  of  a  single  point,  and  Sa©t  is 
x€X 

the  pixel  value  of  the  single  point  image. 

Particularly  nice  examples  that  exhibit  the  brevity  and  translucency  of  image  algebra 
code  and  involves  image  summation  are  order  statistics  of  an  image. 

Example  5.  Moments  as  Descriptors  of  Regions,  Reference  13.  For  any  image 
a  =  {(x,  a(x)) :  x  €X},  with  XC  Z2,  moments  of  order  pq  are  defined  as  follows: 

mpq  =  E  Sipjqa(i,j) 

and  central  moments  as  /cq  =  £  £  (i — i)p(j— j)qa(i,j)  where  i  =  —22- ,  j  = 

i  j  mgo 

The  image  algebra  translation  of  the  moments  is  simply 

mpq  =  £(ip*jq*a) 

where  i={(x,y,i(x,y)):  i(x,y)  =  x  ,  (x,y)€X}  and  j  ={(x,y j(x,y)):  j(x,y)  =  y  ,  (x,y)€X). 
Thus 

mM  =  E.,  .and  J_ -filial 

E»  E. 
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Defining  the  mean  images  i  and  j  by 


the  nonzero  central  moments  are  then  given  by  the  following  translation: 

/ipq  -  E[(i-i)p*(j-j)q*aj 

6.  PARAMETERIZED  TEMPLATES 

Let  X,  Y  be  coordinate  sets  and  P  a  non-empty  set.  A  parameterized  F  valued  tem¬ 
plate  from  Y  to  X  with  parameters  in  P  is  a  function  of  form 

t :  P  — ♦  (Fx)y 

Here  we  define  tp  «  t(p).  Thus  for  each  p  €  P,  tp  is  an  F  valued  template  from  Y  to  X. 
Again,  in  order  to  simplify  notation,  we  define  tpj,  =(tp)y. 

The  set  P  is  called  the  set  of  parameters  and  each  p€P  is  called  a  parameter  for  t. 

Thus,  a  parameterized  F  valued  template  from  Y  to  X  gives  rise  to  a  family  of  regular 
F  valued  templates  from  Y  to  X  namely  {  tp  :  p€P  }.  The  following  two  examples  should 
help  clarify  these  notions. 

Example  6.  The  Kirsch  Edge  Detector,  References  14  and  11.  The  standard  formulation 
of  the  Kirsch  edge  detection  algorithm  is  to  replace  each  pixel  a(y)  of  the  input  image  a  by 

c(y)  =  max{l,  max{  I  5(aj  +  ai+I  +  a,+2)  -  3(8^  +  ...  -I-  ai+7)  I :  i  =  1,2,  ...,  8  }},  where  the 
addition  of  subscripts  is  mod  8  and  the  aj’s  denote  the  following  eight  neighbors  of  a(y): 


a4 

*3 

*2 

a5 

/  \ 
a(y) 

al 

a6 

*7 

*8 

The  image  algebra  expression  representing  Kirsch  algorithm  is  given  by 
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1V(  4  I  a®ts|  ), 

i-i 

where  1  denotes  the  unit  image  and  tj  is  defined  as  follows.  For  i  =  1,2,  ...,  8  define  the 
parameterized  template  tj  by 

V  »  {(x,tj>y(x)):tijy(x)=5  if  x=xj  ,  xi+1  ,  xi+2  ,  tijr(x)=3  if  x  =  xi+3  ,  ...,  xi+7  ,  else  tij((x)=0}  , 
where  the  Xj’s  denote  the  elements  of  5(t-1>y)  as  shown: 


= 


X4 

X3 

x2 

X5 

/  \ 
y 

V  / 

X1 

x6 

X7 

x8 

Example  7.  Image  Magnification.  Suppose  XCR2isanmXn  array,  Y  =  Z2,  P  =  {  p  : 
p=^Xo,k),  where  Xq€  X,  k  a  positive  integer  },  and  a  an  image  on  X  Given  a  pair  of  real 
numbers  r  *  (r1)r2))  define  [r]  s  ([r1],[r2]),  where  [rs]  denotes  truncation  of  rj  to  the  nearest 
integer.  For  each  y  G  Y  and  p=(x0,k),  define  tp^(x)  =  1  if  x  —  [(y— Xo)/k  4-Xq],  and  tpjr(x) 
=  0  otherwise.  Then  b  =>  a  ©  tp  represents  the  magnification  of  a  by  the  factor  k  about  the 
point  X(,.  Thus,  once  this  parameterized  template  has  been  defined,  all  a  potential  user  of 
this  template  needs  to  supply  is  the  magnification  factor  k,  the  point  about  which  to  magnify 
the  image,  and  -  in  order  to  retain  ail  the  information  -  declare  b  to  be  of  at  least  dimension 
kmX  kn.  This  example  also  shows  how  a  template  transformation  is  capable  of  changing  the 
size  of  an  image.  F urthermore,  note  that  from  the  definition  of  the  template  it  follows  that 
$(tP(y)  =  {  x  :  x  =  [(y  -  x,))/k  +  Xq]  }.  In  Figure  6  below,  the  image  on  the  right  represents 
the  magnification  of  the  image  on  the  left  by  a  factor  of  2. 
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/  \ 
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Subtraction,  division,  minimum,  scalar  multiplication,  etc.,  can  be  defined  from  these  basic 
operations  in  a  straightforward  manner.  Thus,  for  example,  t—  a  is  defined  by  (t—  s)y  = 

*y-V 

The  operations  ©,  0  ,  and  ©  between  images  and  templates  generalize  to  operations 
between  templates.  In  particular,  if  t  is  a  real  or  complex  valued  template  from  Y  to  X  and 
a  is  a  real  or  complex  valued  template  from  X  to  W,  then  we  define  the  template  r»i®t 
from  Y  to  W  by  defining  the  image  function  ry  by 

Py(W)=xOCty(X)'8x(W)’  WherC  W  6  W 

In  order  to  compute  the  weights  ry(w)  it  is  usually  not  necessary  to  sum  over  all  of  X 
but  only  a  certain  subset  of  X  In  particular,  given  y,  then  for  each  w  €  W  we  define  the 
set  S(w)  =  {x  €X :  x  €  S(ty)  and  w  €  S(sx) }.  Then,  since  ty(x)sx(w)  =  0  if  x  £  S(w),  we  have 
that 

ry(w)  ~  £  ty(x)sx(w)  , 
yV  ’  x€S( w)  yW  xv 

where  we  define  £  tv(x)  s_(w)  =0  whenever  S(w)  =  0. 

x€S(w)  y  *v  ’ 

The  operations  s  0  t  and  s©  t  are  defined  in  a  similar  fashion.  Here  we  suppose  that 
t  €  (R^)Y  and  s  €  (R^)*  and  define  S^/w)  ==  {x€X:  x^S^tyJandwES^sJ}.  Then 
r=s  0  t  €  (IC)Y  is  defined  by 

r»_*sV>(x)+,«(w)' 


where  we  define 


V 


ty(x)+sx(w)  =  —oo  whenever  S^w)  =  0. 
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Similarly,  r  =s©  t  €  (R^)Y  is  defined  by 

r'(w)“eY(.)Mx)',*(w)' 


where  we  define  V  ty(x)  •  sx(w)  =  — oo  whenever  S^w)  =  (£). 

It  follows  from  these  definitions  that  S(ry)  =  {w  6  W:  S(w)  5^  0},  and 
5oo(ry)={w€W:Soo(w)^0}. 

The  complementary  operations  Q  and  @  are  defined  by 

s  Q  t  =  —  (— s  Q  — t) 


i@  t  =  — (s©— t), 


respectively. 

Initially,  these  definitions  seem  to  be  fairly  complex.  The  following  examples  serve  to 
clarify  these  definitions  and  should  provide  a  better  understanding  as  to  how  composition  of 
templates  is  accomplished. 

Example  9.  Template  Convolution.  Let  s  and  t  be  the  translation  invariant  templates 
defined  in  the  previous  example.  Then 


and  if  s,  t  €  (R*)*,  then 
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Template  composition  and  decomposition  are  the  primary  reason  for  introducing  opera- 
tions  between  generalized  templates.  Composition  and  decomposition  of  templates  provides 
a  tool  for  algorithm  optimization.  For  instance,  if  s  and  t  are  as  in  the  example  above  and  r 
==  s  ©  t,  then  computation  of  a  ©  r  =  a  ®  (s  ©  t)  by  (a  ©  s)  ©  t  uses  6  local  multiplica¬ 
tions  instead  of  9. 

In  general,  if  r  is  an  nxn  template,  and  s  and  t  are  decompositions  of  r  into  IX  n  and 
nX  1  templates,  respectively,  then  the  computation  of  a  ©  r  by  (a  ©  s)  ©  t  uses  2n  multi¬ 
plications  instead  of  n2.  General  methods  for  template  decomposition  and  applications  of 
decompositions  to  algorithm  optimization  can  be  found  in  Reference  7. 
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SECTION  IV 

MULTIVALUE  OR  MULTIDATA  IMAGE  ALGEBRA 


1.  HETEROGENEOUS  ALGEBRAS  AND  HETEROGENEOUS  VALUE  SETS 


Intuitively,  an  algebra  is  simply  a  collection  of  non-empty  sets  together  with  a  finite 
number  of  operations  (rules)  for  transforming  one  or  more  elements  of  the  sets  into  another 
element  of  one  of  the  sets.  In  this  sense,  the  mathematics  of  image  processing  forms  an  alge¬ 
bra  as  it  involves  operations  on  and  between  elements  of  different  sets  such  as  operations  on 
and  between  images,  operations  between  images  and  templates,  operations  between  tem¬ 
plates,  etc. 


One  of  the  main  tools  of  algebra  are  isomorphisms.  If  two  algebraic  structures  are  iso¬ 
morphic,  that  is,  if  there  exists  a  one-to-one,  operation  preserving  mapping  of  one  onto  the 
other,  then  they  provide  two  different  viewpoints  of  the  same  situation.  The  idea  is  that  the 
more  ways  one  has  to  look  at  a  problem  the  better  chance  there  is  of  solving  it. 


To  make  the  notion  of  an  algebra  mathematically  precise,  we  define  an  algebra  as  pair 
(A,0)  in  which 


1. 


2. 


A  =■  {A^}  is  a  family  of  non-empty  sets  of  different  types  of  elements  and  the  sub¬ 
scripts  j  are  members  of  some  common  indexing  set  J,  and 

0  =  {  o-, }  is  a  set  of  finitary  operations,  where  each  operation  ok  G  0  is  a  mapping 


with  each  A^^EA  and  II  A^  denotes  the  cartesian  product 


i?A=A^x  ‘ ’ xA»*  =  {(av  •  •  •  ’%) :  \  6  \  >• 


The  operation  ok  is  unary  if  n  =  1,  binary  if  n  =  2,  ternary  if  n  =  3,  etc.  The  elements 
Aj  of  A  are  called  the  sets  of  operands  of  A.  Also,  whenever  the  set  of  operations  0  is 
tacitly  understood,  it  is  customary  to  let  A  denote  the  algebra  (A,0). 

Suppose  (A,0)  and  (B,Q)  are  two  algebras  with  A={Aj}and  B  =  {B-,}.  Then(B,i2) 
is  called  a  subalgebra  of  the  algebra  ( A,0 )  if  flC  0  and  for  each  B-,€B  there  exist  A^EA 
such  that  B;  C  Aj . 
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An  algebra  which  has  only  one  set  of  operands  is  called  a  homogeneous  algebra,  while  an 
algebra  with  more  than  one  set  of  operands  is  called  a  heterogeneous  algebra.  For  example, 
the  real  numbers  R  together  with  the  operations  of  addition,  multiplication,  and  maximum 
constitutes  a  homogeneous  algebra  that  falls  under  the  general  class  of  algebras  known  as 
lattice-ordered  rings,"  Reference  15.  On  the  other  hand,  the  algebra  A  defined  by  A  = 
{R,RX,(RX)X  }  and  0  =  {  +,*,«,©  }  constitutes  a  heterogeneous  algebra  which  is  a 
subalgebra  of  the  image  algebra  defined  in  this  paper.  Various  subalgebras  of  the  image  alge¬ 
bra  are  related  in  terms  of  isomorphisms  to  well-known  algebraic  structures  such  as  linear 
algebra  and  mathematical  morphology,  References  7  and  3.  It  is  these  relationships  of 
subalgebras  of  the  image  algebra  to  other  well  established  mathematical  structures  or  con¬ 
cisely  defined  areas  of  image  processing  which  endow  the  image  algebra  with  its  versatility 
and  power.  All  theorems,  relationships,  and  "tricks-of-the-trade"  associated  with  these  struc¬ 
tures  can  be  interpreted  and  exploited  within  the  language  of  the  image  algebra  via  these 
relationships.  For  example,  in  Reference  7  we  demonstrated  how  several  of  these  relation¬ 
ships  can  be  applied  to  the  problem  of  developing  systematic  techniques  for  the  optimization 
and  derivation  of  parallel  algorithms.  The  main  purpose  of  this  section,  however,  is  to 
extend  the  image  algebra’s  capability  to  include  operations  for  the  manipulation  of  mul¬ 
tivalued  images. 

n 

Suppose  F  =  nF;,  where  each  F:  is  a  value  set  whose  set  of  Unitary  operations  is 
i— 1 

The  set  of  naturally  induced  operations,  0,  on  F  is  defined  as: 

Q 

0={  o£  II  0-, :  the  coordinates  of  o  have  the  same  arity  }. 

i— i 

Thus,  if  o  — (o1( . ,on)  and  o-,  is  a  binary  operation  on  Fj  for  some  i  between  1  and  n,  then 

o€0  only  if  each  of  the  remaining  coordinates  o}  is  a  binary  operation.  The  operation  o  is 
unary  if  its  coordinates  are  unary  operations,  binary  if  its  coordinates  are  binary  operations, 
etc. 

If  f=(fj . ,fn)GF  andg=(g1( . ,gn)€F  and  o=(o1( . ,on)€0  is  binary,  then  we 

define 

fog=(f,Oigi, . ,f„oBgn) 

n 

We  now  extend  the  notion  of  a  value  set  to  include  sets  of  form  F  =  OF:  where  each  F=  is  a 

i-l 
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value  set  and  the  operations  on  F  are  the  naturally  induced  operations.  It  follows  that  every 
value  set  is  an  algebra. 

n 

If  F  =  n  F;  and  F:=Fs  for  all  i,j  =  l,2,....,n,  then  F  is  called  a  homogeneous  value  set, 

otherwise  F  is  called  a  heterogeneous  value  set.  If  F  is  homogeneous  and  o=(ov . ,on)  is 

an  operation  on  F  with  Oj=Oj  for  all  ij  =  1,2,... .,n,  then  o  is  called  a  homogeneous  opera¬ 
tion,  otherwise  o  is  called  a  heterogeneous  operation.  Whenever  o  is  homogeneous,  then  it 
is  customary  to  use  o-,  to  denote  the  operation  o.  For  instance,  if  F  =  FtX  F2X  F3  and 
F,  =  R  for  i  =  1,2,  and  3,  then  F  =  R3  and  F  is  homogeneous.  Furthermore,  if  0; ={+,*, V}, 
then  +=(+,+,+)  and  V=(V,V,V)  are  homogeneous  operations  while  o  =(+,*,  V)  is  a  hetero¬ 
geneous  operation.  Applying  these  operations  to  the  elements  a=(a1,a2,a3)  and  b=(b1,b2,b3) 
of  R3  yields 

a+b=(a1+b1,a2+b2,a3+b3),  aVb=(a! Vb^Vb^agN/bg),  and 


aob=(a1+b1,a2*b2,a3Vb3). 


2.  OPERATIONS  ON  MULTIVALUED  IMAGES 

n 

Let  a  be  am  F  valued  image  on  X  If  F  *  II  Fj  and  n  >  1,  then  a  is  called  a  mul- 

i-1 

tivalued  or  multidata  image.  We  distinguish  between  two  types  of  multivalued  images.  If 
a€Fx  and  F  is  heterogeneous,  then  a  is  called  a  heterogeneous  multivalued  image,  and  if  F 
is  homogeneous,  then  a  is  called  a  homogeneous  multivalued  image. 

n 

Given  n  sets  Sj ,  •  •  •  ,  Sn,  then  the  function  p, :  II  Sj  — ►  S:,  where  1  <  j  <  n,  defined  by 

Pj(sj ,  •  •  •  ,  Sj ,  •  •  •  ,sn)  ~Sj,  is  called  the  projection  onto  the  jth  coordinate  or  the  jth  coordi¬ 
nate  projection.  Projection  functions  play  an  important  role  in  the  manipulation  of  mul¬ 
tivalued  images.  This  is  due  to  the  fact  that  if  a  €  Fx  is  a  multivalued  image  with 

n  n 

F  =  JI  Fj,  then  -  since  FX=(JI  F,)x  -  the  image  a  can  always  be  viewed  as  a  "stack"  of  n 

single-valued  images  a=(a1,a2,  .  .  .  ,a„)  where  the  ith  coordinate  image  of  the  stack  is 
defined  as 

=Pi(a)  =  {(x,a,(x)) :  a,(x)  =p-,(a(x)) }. 
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Thus,  *€(F{jX. 

A  typical  LANDSAT  image  a  of  n-spectral  bands  provides  a  simple  example  of  a  mul- 

n 

tivalue  image  that  can  be  viewed  as  a  stack  of  single  valued  images.  Here  F  =  II  Fj  =  Rn 

i-i 

and  a(x)=(a1(x), . .a^x)),  with  each  aj(x)£R. 

The  unary  and  binary  operations  on  multivalued  images  are  the  operations  induced  by 

D 

the  set  0  of  operations  of  the  underlying  value  set  F.  For  instance,  if  F  =  ITFj,  a  = 

i— x 

{(x,a(x))  :  a(x)  =  (a^x), . an(x))}  and  b  -  {(x,b(x)) :  b(x)=(bx(x) . bn(x))}  are  F  valued 

images  on  X,  and  o=(ox, . on)€  0  a  binary  operation  on  F,  then 


k  ob  ={(x,c(x)) :  c(x)  =(a1(x)  b^x), . ,a„(x)  on  bn(x)) } 


or,  equivalently, 


If  o  is  unary,  then 


or,  equivalently, 


aobs(a1o1bl, . ^  oBbD). 


o(a)  s{(x,c(x)) :  c(x)  =(o1(a1(x)), . ©nMx)))}. 


«(*)=(®x(»i) . onM)- 


For  example,  if  F  =R"  and  o  is  a  binary  homogeneous  operation  with  o-,  =  +,  then 

a+b  =(a!  +b1( . ,a„  +bn)  «{(x,c(x)) :  c(x)  =(a,(x) +bj(x), . a„(x)  +bn(x)) } 

Similarly,  if  o  is  a  unary  homogeneous  operation  with  Oj  =  sin,  then 

sin(a)  =(sin(a1) . ,sin(an))={(x,c(x) :  c(x))  =(sin(a1(x)), . ,sin(an(x)))}. 

and  if  Oj  =  X><  then 

X>b(»)  =  {(x,c(x)):ci(x)  *  1  if  a,(x)  >  bj(x),  otherwise  c^x)  =  0,  x  6  X} 


It  readily  follows  from  these  examples  that  if  F=Ra,  then  the  operations  defined  for  R 
valued  images  extend  in  a  natural  way  to  RB  valued  (vector  valued)  images.  For  example,  if 
a  and  b  are  Rn  valued  images  on  X  then  the  dot  product  is  defined  as 


a*b  =  Ea(x)bl(x) 
xcX 


where  bl(x)  denotes  the  transpose  of  b(x).  The  notion  of  scalar  multiplication  and  addition 
are  extended  to  vector  multiplication  and  vector  addition  in  a  similar  fashion.  In  particular, 
if  b  — {(x,b(x)):bi(x)=kj  VxGX},  and  k  is  the  vector  k  =(k1(  .  .  .  ,  kn)  G  R“,  then  we 
define 

ka  s  b*a  =  {(x,c(x)) :  Cj(x)  =  kiai(x),  x  G  X}. 


and 


'k  +  a  =  b+a  =  {(x,c(x)) :  Cj(x)  =  kj+a^x),  x  G  X} 

In  the  particular  case  where  b  is  the  constant  vector  valued  image  b  = 
{(x,b(x)):bj(x)*>k  VxGX  and  l<i  <n},  we  have  that  k  =»(k,...,k),  and  we  define 

ka  3  ka  s  b*a  and 
k+a  =  k+a  =  b+  a. 

Obviously,  these  notions  extend  to  F  =  C“  valued  images  and  vector  valued  images  in 
general.  In  comparison  to  the  next  set  of  operations,  the  operations  just  discussed  are  not 
"new"  operations  but,  as  in  the  case  of  single  valued  images,  the  naturally  induced  operations 
of(F.O). 

n  k 

Let  F  =  II  Fj  and  F'=  II  F'=  be  two  value  sets.  If  n  <  k,  then  any  function  g :  F  — ►F'  is 

i-1  i-l 

called  a  data  splitting  function.  If  n  >  k,  then  g  is  called  a  data  fusion  function.  In  either 
case,  g  is  of  the  form  g=(gi,g2,  ■  ■  ■  ,  St)  where  g(  =  Pjg.  Thus,  the  function  g:  R  — ►  R2  is 
defined  by  g(r)  =  (cos  r,  sin  r)  is  a  value  splitting  function,  while  the  function  f:  R2  — ►  R 
defined  by  ffo.r-j)  =  rj+r2  provides  an  example  of  a  data  fusion  function. 

Two  elementary  fusion/splitting  functions  of  prime  importance  are  the  (generalized) 

n  k 

projection  and  injection  functions.  Let  F  =  n  F:  and  F'=  IT  F:  ,  where  F;  denotes  the 

i-l  m— 1  w  Jm 

jmth  factor  of  F.  Then  the  projection  Pj,^..^ F  — ►  F',  which  deletes  n— k  components 

D 

from  II  Fj ,  is  defined  by 
i— 1 


’  ^2  >  ■—  >  ^n)  —  (fj, » fj, . .  *  where  fjm€Fjm. 

For  F  a  homogeneous  value  set  we  define  the  injection  function 
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q  :  Fk  -►  F,  by  q(r)  =  (r,  r,...,  r) 

The  ith-coordinate  injection  qj  :  F  — ►  F  is  defined  as  qj  =  qp,.  Thus  q;  replaces  all  the 
coordinate  values  of  a  point  by  the  value  of  the  ith-coordinate,  namely 

q»(rl»  r2i  •  •  •  i  ri>  •  •  •  >  rn)  =  (ri>  ri>  •  •  ■  >  ri) 

The  basic  operations  on  single  and  multivalued  images  together  with  the  injection  and 
projection  functions  can  be  combined  to  form  useful  image  processing  operations  of  arbitrary 
complexity.  Projection  functions,  for  example,  can  be  used  to  reduce  a  multivalued  image  of 
n  stacks  to  a  single  valued  image  or  a  multivalued  image  of  k  stacks,  where  k  <  n.  For 
instance,  if  a£Fx  with  F  =  Rn,  then  we  define 

n 

£a=Epi(a)  -  »i  +  »2  4- ....  +  a„ 

and 

Va  =  VPi(a)  =  ax  V  ajj  V  ....  V  a„ . 
i-l 

As  another  example  of  obtaining  new  operations  by  combining  previously  defined  opera- 

a 

tions  with  projections  and  injections,  consider  the  case  where  F  =*  n  F=  and  F=  *  R.  Then 

i-l  J 

we  define  the  jth-coordinate  maximum  and  minimum  of  two  F  valued  images  as 
aVljb  =  {(x,c(x)):c(x)  =  a(x)  if  aj(x)  >  bj(x),  otherwise  c(x)  =  b(x)  } 

and 

aAljb  =  {(x,c(x)):c(x)  =  a(x)  if  aj(x)  <  bj(x),  otherwise  c(x)  =  b(x)  }, 

respectively.  Using  injections  and  projections,  these  operations  can  be  derived  from  the  pre¬ 
viously  defined  operations.  In  particular, 

aVljb  =  a*qj[x>b(a)]  -I-  b*qj[x>b(a)] 

and 

aAljb  =  a*qj[x<b(a)]  +  b*qj[x<b(a)]. 

In  order  to  illustrate  the  use  of  multivalued  image  operations,  we  present  a  typical  appli¬ 
cation  example. 
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Example  10.  Directional  Edge  Detection.  The  output  of  this  directional  edge  detection 
scheme  is  a  two-valued  "edge"  image  in  which  each  pixel  has  an  intensity  value  as  well  as 
one  of  eight  possible  directional  values.  In  this  particular  scheme,  a  grey  scale  image  is  con¬ 
volved  with  the  following  four  3x3  edge  masks  (templates),  with  each  mask  corresponding  to 
two  possible  direction. 
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Figure  7.  The  Four  Directional  Edge  Detection  Masks 

The  resulting  four  images  are  then  fused  to  form  a  single  2-valued  image  b.  Data  fusion  is 
accomplished  by  assigning  to  the  resultant  pixel  the  value  of  the  largest  magnitude  of  the 
corresponding  pixels  in  the  four  images  and  either  assigning  the  direction  9  associated  with 
the  mask  of  the  convolved  image  if  the  specific  pixel  value  of  the  convolved  image  is  positive, 
or  6  +  180°  mod  360°  if  the  value  is  negative.  Thus,  each  pixel  of  b  has  both  a  magnitude 
and  a  direction  associated  with  it.  It  is  customary  to  use  the  integers  0  through  7  to 
represent  the  eight  directions  0°  through  315°,  respectively.  The  addition 
"0  +  180°  mod  360°"  then  becomes  addition  modulo  eight,  namely  "i  +4  mod  8". 

The  image  algebra  translation  of  this  algorithm  is  as  follows.  Let  f:R  — ►Rx  Z8  be 
defined  by  f(r)  =  (I  rl,4x<o(r))»  then 

i.  =  (0,i)  -I-  f(a@mi),  i=0,...,4  and 
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ii.  b=(Vli)  a, 
i-o 


The  next  three  figures  provide  a.  pictorial  example  of  this  algorithm. 


Figure  8.  The  Input  Image  a 
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3.  MULTIVALUE  TEMPLATES  AND  MULTIVALUE  TEMPLATE  OPERATIONS 

n 

If  F  =  n  F,  and  n>l,  then  t€(Fx)Y  is  called  a  multivalue  or  multilevel  template  from 
i— 1 

Y  to  X  Analogous  to  multivalued  images,  a  multilevel  template  t  can  be  thought  of  as  a 
stack  of  single  valued  templates  t=(t1,t2, . ,tn),  where  each  ith-coordinate  tj  €(FX)Y  is 
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defined  by  tj  =  p-,t.  Here  p-,  denotes  the  ith  projection  map  p-, :  F  — ►  Fj  and  the  weights  of  fc-,  y 
are  given  by  p-,ty(x). 

Operations  between  multivalued  images  and  multilevel  templates  are  natural  extensions 
of  previously  defined  operations  between  single  valued  images  and  templates.  Suppose 
a€Fx  and  t€(Fw)Y,  where  W  and  X  are  subsets  of  the  same  euclidean  space.  Let 

o=(o1( . ,on),  where  each  o-,  is  an  operation  between  Fj  valued  images  on  X  and  Fj  valued 

templates  from  Y  to  W.  Then  aot  is  defined  as 

aots^o^, . 

To  illustrate  this  concept,  consider  the  case  where  F  =R3  and  o—(®,  Q  ,  ©  ).  Then 

aot s(&| © , &2  E3  t2,ag©t3) 

Also,  if  o  is  homogeneous,  say  o  *(©,©,©),  then  -  following  our  earlier  convention  -  we 
set  o  =©  and  define 

a©  t  =(at  ©  tj ,  aj©  to ,  aj©  tg) 

Thus,  a  multilevel  template  can  have  different  configurations  and  can  operate  differently  on 
the  different  levels  of  a  multivalue  image.  Particular  application  examples  of  operations  on 
multivalued  images  can  be  found  in  References  16  and  17. 

As  a  final  observation  we  note  that  if  F  and  F*  are  two  value  sets  with  F  F'  and 
o:FxF'-*F  a  binary  operation,  then  the  operations  between  F  valued  images  and  F 
valued  templates  can  be  generalized  to  operations  between  F  valued  images  and  F'  valued 
templates.  For  example,  if  F  is  a  vector  space  over  the  field  of  scalars  F'  (e.g.,  F=Rn  and 
F'=R),  a€Fx  and  t€(F'w)Y,  where  W  and  X  are  subsets  of  the  same  euclidean  space, 
then  we  can  define 

c  =a©t  by  c(y)  =  £  tJx)-a(x),  where  y  €  Y. 

xexTls^) 

Thus  c  is  a  vector  valued  image  with  values  in  F. 

Obviously,  if  F  is  a  vector  lattice  over  F',  then  we  can  define  a  Q  b  and  a©  b  in  a 
similar  fashion.  In  the  case  where  a€F'x  and  t€(Fw)Y,  the  operation 
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c=a©t  defined  by  c(y)  =  Ij  a(x)-ty(x) 

x€xns(ty) 

turns  the  scalar  valued  image  a  into  a  vector  valued  image  c.  Hence  templates  can  be  used 
to  change  not  only  the  dimensionality,  shape  and  size  of  an  image,  but  also  its  value  type. 


4 
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SECTION  V 

DISCUSSION  AND  CONCLUSION 


This  document  presents  an  analysis  of  the  Singer-Kearfott  algebra  and  an  overview  of 
the  AFATL  Standard  Image  Algebra.  As  we  consider  this  paper  an  introduction  to  the  sub¬ 
ject  and  in  order  to  keep  the  length  of  this  paper  within  reasonable  limits,  we  have  purposely 
restricted  application  examples  to  a  few  simple  and  well-known  image  transforms.  Proofs 
that  the  algebra  is  capable  of  expressing  all  image- to- image  transformations  have  been 
presented  elsewhere,  References  6  and  9.  In  previous  reports  to  AFATL  we  demonstrated 
the  full  power  of  the  image  algebra.  In  particular,  we  have  shown  how  the  image  algebra 
encompasses  such  structures  as  linear  algebra,  polynomial  algebra  and  the  minimax  algebra 
of  economics  and  operations  research,  Reference  18,  and  how  these  relationships  can  be 
exploited  for  useful  applications  in  image  processing. 

At  first  glance,  it  may  seem  that  the  image  algebra  lends  itself  well  only  to  expressing 
parallel  type  of  image  processing  operations.  However,  it  turns  out  that  the  algebra  is  quite 
capable  of  expressing  such  sequential  processes  as  chain  encoding,  Reference  19,  in  a  more 
compact  and  translucent  fashion  than  is  possible  within  current  higher  level  languages,  Refer¬ 
ence  20.  It  has  been  our  experience  that  image  processing  algorithms  requiring  a  variety  of 
routines,  including  purely  sequential  ones,  when  coded  in  Image  Algebra  Fortran  instead  of 
Fortran  have  always  resulted  in  significant  code  reduction,  Reference  21. 

In  conclusion,  the  image  algebra  in  its  present  form  provides  a  comprehensive  and 
unified  algebraic  structure  for  the  representation  of  image  to  image  operations.  However, 
although  image  algebra  operations  on  images  can  be  used  to  extract  statistical  and  geometric 
measures  or  image  representations  from  images  such  as  centroids,  Euler  number,  and  chain 
codes,  we  have  made  no  serious  attempts  to  extend  the  algebra  to  the  symbolic  domain.  In 
particular,  "high  level"  image  operations  which  employ  tools  from  such  diverse  areas  as 
knowledge  representation,  graph  theory,  and  surface  representation  have  not  been  con¬ 
sidered.  Furthermore,  the  mathematics  associated  with  the  image  algebra  and  its  implicar 
tions  to  image  processing  is,  in  itself,  largely  unchartered  territory.  Thus,  the  image  algebra 
in  its  current  state  is  not  a  finished  product,  but  a  continuously  evolving  mathematical 
theory  concerned  with  the  unification  of  image  processing  tasks. 
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